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ABSTRACT 



This paper presents a multifaceted approach for 
investigating the impact of the Success for All (SFA) program on children's 
reading proficiency. The approach advocated features case study research and 
uses document analysis, structured observation, interviews, and 
questionnaires to collect data. The research design was successfully field 
tested in two Mississippi elementary schools, one in a rural community and 
one in a university community. Strategies presented in this paper can be used 
by experienced or novice researchers to study the SFA reading program in an 
elementary school. Student and teacher questionnaires are attached. 
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Abstract: This paper presents a multi-faceted approach for investigating the impact of 
the Success For All program on children's reading proficiency. The approach 
advocated in the paper features case study research and uses document analysis, 
structured observation, interviews, and questionnaires to collect data. The research 
design was successfully field-tested in two Mississippi elementary schools--one 
located in a rural community and one located in a university community. Strategies 
presented in this paper can be used by either experienced or novice researchers to 
study the SFA reading program in an elementary school. 
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BEST COPY AVAILABLE 



Touted as a marketed solution for addressing children’s low reading 
performance, Success for All (SFA), developed by Dr. Robert Slavin an his associates 
at Johns Hopkins University, is one of the leading packaged reading programs on the 
national market. Although its developers originally designed the program to benefit 
students who rank in the lowest 25% in reading achievement, it's presently being 
adopted and used in elementary schools throughout the United States as a complete 
reading curriculum for all students in kindergarten through grade six (Slavin, 1998). 

Does SFA produce children who learn to read and then become readers who 
read for pleasure? Unfortunately, results are mixed. Although the program appears to 
increase children’s reading achievement as narrowly measured by standardized tests, 
it's uncertain whether the program produces children who become proficient learners 
because they love to read. Very little comprehensive research concerning SFA has 
been conducted to date (Blendinger & Wells, 2000). 

This paper presents a multi-faceted approach, featureing case study research, 
for investigating the impact of the SFA program on children's reading proficiency. 
Document analysis, structured observation, interviews, and questionnaires are used to 
collect data. Our research design we developed was successfully field-tested in two 
Mississippi elementary schools— one located in a rural community and one located in a 
university community. The design can be used by either experienced or novice 
researchers to study the SFA reading program in action (Wells, 2000). 

Case Study Research 

We believe case study research provides one of the best approaches available 
for comprehensively investigating the impact of the SFA reading program in an 
elementary school using a variety of measures ranging from standardized test scores 
to parent attitudes. According to Gall, Borg, and Gall (1996), a case study is done to 
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shed light on a phenomenon, which is the processes, activities, or persons of interest 
to the researcher. A case is a particular instance of the phenomenon. 

In a case study, a substantial amount of data can be collected about the specific 
case (or cases) selected to represent the phenomenon. These data can be in the form 
of words, images, or artifacts. Case study research involves watching people in their 
own territory and interacting with them in their own language, on their own terms. 
Quantitative data may be collected as well qualitative data. Often data are collected 
over an extended time period using several methods of collection. 

Case study research typically involves fieldwork in which the researcher 
interacts participants studied in their own natural settings. Researchers generally do 
case studies for one of three purposes: (a) to produce detailed descriptions of a 
phenomenon, (b) to develop possible explanations of the phenomenon, or (c) to 
evaluate the phenomenon. 

In a case study whose major purpose is description, the researcher attempts to 
depict a phenomenon and conceptualize it. A good depiction provides what is 
commonly referred to as thick description of the phenomenon: that is, statements that 
recreate a situation and as much of its context as possible, accompanied by the 
meanings and intentions inherent in that situation. Researchers add depth to their 
descriptions by searching for the presence of themes: the salient, characteristic 
features of a case. 

Often the major purpose of case study research is to provide explanations for 
the phenomenon studied. Explanations may be seen as patterns, meaning that one 
type of variation observed in a case study is systematically related to another observed 
variation. If the researcher does not claim that one variation has a causal effect on the 
other, it’s described as a relational pattern. If causality is claimed, it's a causal pattern. 

Evaluation of the phenomenon investigated can also be the major purpose of 
case study research. If making judgments is the chief reason, the researcher should 
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create a thick description of the phenomenon being evaluated and identify salient 
constructs, themes, and patterns. Case studies whose purpose is evaluation are being 
done with increasing frequency. 

In using case study research to investigate the SFA reading program, we 

suggest that the researcher address all three purposes: description, explanation, and 

evaluation. In addition, the researcher should develop a succinct demographic profile 

of the school studied. An excerpt from one our SFA case studies showing what we 

mean by "demographic profile" follows: 

The elementary school located in the rural community enrolled approximately 
600 students in kindergarten through grade six during the 1998-99 school year. 
The school is named Alpha Elementary School in this study for purposes of 
anonymity. The school district in which the school is located serves a student 
population that is 84 percent African-American and 16 percent White. Seventy- 
five percent of the students are eligible for free or reduced lunches and 14 
percent are identified for special education services. The district does not offer 
any advanced placement courses and has a 73 percent graduation rate. The 
total per pupil expenditure is $4,421 (Wells, 2000). 

The forthcoming sections of the paper focus attention on how document analysis, 

structured observation, interviews, and questionnaires can be used to investigate the 

impact of the SFA program on children’s reading proficiency. 



Document Analysis and Field Observation 

In investigating the impact SFA has on children's reading proficiency, we 
suggest that the researcher starts by examining written documents such as the 
program's prospectus. Manning and Cullum-Swan (1994) state that to fully understand 
a document or record, the researcher should study the context in which it was 
produced: the author's purpose in writing it, working conditions, and audience. SFA 
documents indicate the following: 

Prescriptive curriculum should be closely followed to teach reading. 

Reading is taught in a 90-minute block every morning. 

Reading instruction is supervised and coordinated by a facilitator. 

Cross-grade ability grouping is used for instruction. 

Children are placed homogeneously across grade levels. 
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Children of similar ability are taught in a whole-class setting. 

Children are assessed and regrouped as necessary. 

Tutoring is provided children who need extra assistance. 

Parents are involved in their children's reading. 

Family support team should be an integral part of the program. 

We used the above-mentioned information as a framework to guide our observations 
and then recorded what we observed in the form of field notes. According to Yin 
(1984), field notes are best made using “continuous recording” techniques. 

Continuous recording involves scripting all the behavior of the targeted setting and 
subjects for a specific observation interval. To determine whether SFA procedures for 
implementation are being followed as advocated by the program's developers, we 
recommend making at least 10-15 hours of field observation. 

School and classroom records should be examined to collect data for 
framework components that may be difficult to observe, such as parent involvement or 
family support team activity. Document analysis also provides the means for 
investigating how SFA affects children's reading scores on standardized tests (e.g., 
Iowa Tests of Basic Skill) in terms of normal curve equivalency (NCE) scores. NCEs 
are equal-interval scores, derived from national percentile ranks, that result from 
dividing the normal curve into 99 equal units. NCEs are units of measurement used in 
many southeastern states to assess student achievement for accreditation purposes. 
Standardized test scores pertaining to reading achievement are usually available for 
most elementary schools. Test scores for student cohorts (i.e., the same group of 
students as they precede through the various grade levels comprising the school) 
should be compared for two or more years. 

Key Informant Interviews 

To investigate what key informants report about the impact the reading program 
is making, we recommend conducting semi-structured interviews with the principal 
and the SFA facilitator. The “key informant” approach involves interviewing individuals 
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who have special knowledge of the situation. Semi-structured interviewing requires 
asking a series of structured (closed-form) questions and then exploring the key 
informants' responses more extensively using more open follow-up questions to 
obtain additional information (Gall, Borg, & Gall, 1996). 

Sample questions that can be used to guide the interview with the principal are 
as follows: 

1 . What were some of the problems encountered when SFA was initially 
implemented? 

2. How is the program funded? What is the approximate cost per pupil for full 
implementation of SFA? 

3. Does every certified teacher teach reading? 

4. How would you describe what goes on during the 90-minute reading period? 

5. How many certified teachers provide one-on-one tutoring to children 
experiencing difficulty? How many children are tutored? 

6. Do tutors work with the students they are tutoring on lessons they have been 
previously taught so that the material is familiar to them? 

7. How are special education students involved in the SFA program? Has there 
been a reduction in special education placements since implementing SFA? 

8. How are parents involved in the program? 

9. Do students participate in reading celebrations? What are they like? 

10. What are the classroom libraries that are part of the SFA package like? 

1 1 . What testing measures do you use to chart students' gains? 

12. What is the percentage of student movement at eight-week testing periods? 

13. How does SFA mesh with the state's reading and language arts curriculum 
framework? 

14. Is the writing component part of your program? How does it work? 

15. How does SFA's home reading component function? 

16. How does the family support team and the family advisory board work? 
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17. Do you have volunteer listeners? 

1 8. How does SFA's staff development component function? What problems 
have you encountered? 

1 9. How often do SFA officials visit? How else is the program monitored? 

Sample questions that can be used to guide the interview with the SFA facilitator are 
as follows: 

1 . What are your responsibilities as facilitator of the SFA reading program? 

2. What goes on during the 90-minute reading period? 

3. How many certified teachers provide one-on-one tutoring to children 
experiencing difficulty? What is the average number of children receiving 
tutoring? 

4. Do tutors work with the students they are tutoring on lessons they have been 
previously taught so that the material is familiar to them? 

5. How are special education students involved in the SFA program? Has there 
been a reduction in special education placements since implementing SFA? 

6. How are parents involved in the program? 

7. Do students participate in reading celebrations? What are they like? 

8. What are the classroom libraries that are part of the SFA package like? 

9. What testing measures do you use to chart students' gains? 

10. What percentage of student movement is there at eight-week testing periods? 

1 1 . How does SFA mesh with the state's curriculum framework for reading and 
language arts? 

12. Is the writing component part of your program? How does it work? 

13. How does SFA's home reading component function? 

14. How does the family support team and the family advisory board work? 

15. Do you have volunteer listeners? 

16. How does SFA's staff development component function? What problems 
have you encountered? 

17. How often do SFA officials visit? How else is the program monitored? 
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1 8. What is the break down of students by reading proficiency levels? What 
percentage of your student population is reading at or above grade level? 

19. How does the grading system used for the SFA program work? 

Data collected from interviews with the principals and reading facilitators at the 
two school or schools comprising the case should be analyzed for the purpose of 
identifying emerging themes. Verbatim responses to the interviews need to be edited 
and summarized for readability. Care also must be exercised to not change or distort 
content. 



Surveying Attitudes 

Questionnaires should be used to survey the attitudes of teachers, students, 
and parents toward the SFA reading program in particular and reading in general. We 
recommend the following procedures for measuring the attitudes of teachers, students, 
and parents involved in the case: 

1 . Researcher directly administers the questionnaires developed for teachers at 
a brief meeting before or after school. The primary purposes of the teacher 
instrument are (a) to collect demographic information about personal 
characteristics of the respondents in order to establish a teacher profile, and 
(b) to measure attitudes about their involvement in the SFA reading program. 
A copy of the teacher questionnaire we designed accompanies this paper. 

2. Teachers administer the questionnaires developed for students to the 
students in their classrooms in a whole group setting at an unobtrusive time, 
such as just before the school day ends. The primary purposes of the student 
instrument is to measure their attitudes about (a) the SFA reading program in 
particular, and (b) reading in general. Copies of the two student 
questionnaires (one for K-2 students and one for students in grades 3-6) we 
designed accompany this paper. 

3. Teachers send the parent questionnaire developed for parents home with 
students and collect them the next day. The primary purposes of this 
instrument are: (a) to collect demographic information about personal 
characteristics of the respondents in order to establish a parent profile, (b) to 
measure parents' attitudes about their children's involvement in the SFA 
reading program in particular, and (c) to measure parents' attitudes about 
their children's reading habits in general. A copy of the parent questionnaire 
we designed accompanies this paper. 
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Prior to administering the questionnaires, information concerning the purpose of 
the study should be given to each participating teacher. Letters of assent for 
participation of minors, as well as consent forms for parent participation, can be sent 
home with the students. Teachers and parents need to be informed when and how to 
return the forms and questionnaires. Care must be taken to ensure participants that 
their participation in the survey process is voluntary. 

Scales for measuring attitudes vary according to the items on the instruments. 
Some items are measured using two-point scales, other items are measured using 
three-point scales, and still other items are measured using five-point scales. Open- 
ended questions are also part of the instruments. 

We recommend tabulating and describing responses to the instruments' 
questions and statements in terms of frequencies (i.e., number of respondents 
selecting a specific response) and percents. In addition to descriptive measures, chi- 
square "goodness of-fit" tests may be used to determine whether the frequency 
distribution shows a specific pattern. Although observed frequencies will almost 
always differ somewhat from expected frequencies, chi-square tests can determine 
whether the differences are statistically significant or due to chance. An alpha level of 
.01 should be selected to determine statistical significance (Bluman, 1997). 

Responses to the instruments' open-ended questions asking respondents to 
state their opinions about the positive and negative aspects of the SFA reading 
program should be analyzed for the purpose of identifying emerging themes. Themes 
(i.e., grouping data in terms of categories) represent recurring regularities in the data. 

Closing Remarks 

This paper presented a multi-faceted approach for investigating the impact of 
the SFA program on children’s reading proficiency. In closing, it should be noted that 
the research design was developed based on the two Mississippi elementary schools 
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comprising the case studied: a school located in a rural community and a school 
located in a university community. 

There is need for continuing case study research investigating SFA's long-term 
impact on children's reading achievement and habits, such as reading for pleasure. 
Much more comprehensive research must be conducted to determine whether the 
SFA reading program produces children who enjoy reading and is worth the costs 
incurred. 
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teacher attitudes toward the success for all reading program 



The purpose of this brief questionnaire is to collect information concerning teacher 
attitudes toward the Success For All (SFA) reading program. Your participation is 
much appreciated. 

ndicate your answer for each question by checking the appropriate space. 

Race 

African American 

American Indian 

Asian 

Hispanic 

White 

Other 



Year of teaching experience 

0-5 

6-10 

11-15 

16-20 

21-25 

more than 25 



Years of experience teaching reading 

0-5 

6-10 

_ 11-15 

16-20 

T 21-25 

more than 25 



Dominant method of reading 
instruction prior to implementing SFA 

Basal series 

, Language experience 

Literature-based 

Trade books 

Other 



Check the appropriate responses. 

yes no 

I had the opportunity to observe the SFA program in other schools. 

The administration provided me with adequate information about the SFA program. 

I had sufficient time to study the program prior to making a decision about adopting SFA. 

I voted to implement the SFA program at my school. 



Indicate your agreement with each of the following statements by checking the 
appropriate response. 



I received adequate training to teach SFA reading 
prior to its implementation. 
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Follow-up training has been helpful. 

Support from the administration and the program 
facilitator is helpful. 

Children's attitudes about reading have improved. 



I have access to all the resources I need to 
implement the SFA program. 

I have an adequate classroom library stocked with 
books that match the students’ reading levels. 



Students frequently check out books from my 
classroom library. 



Parents are involved in their children’s reading. 
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strongly disagree 



Check the components of the SFA reading program that are implemented at your 
school. 

y es no uncertain 

90-minute reading program 

Children are grouped by reading performance ability. 

Reading class size does not exceed 20 students. 

One-on-one tutoring is conducted by certified teachers. 

Reading class begins by sharing engaging literature with the children. 

Ail students are tested at the end of each 8-week period. 

Reading materials are organized by support staff. 

Do you prefer teaching reading using the SFA approach compared to other methods 
you have used in the past? Please comment on your answer. 



What are some aspects of the SFA program that you especially like? 



What are some aspects of the SFA program that you dislike or would like to change? 
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PARENT ATTITUDES TOWARD THE SUCCESS FOR ALL READING PROGRAM 

The purpose of this brief questionnaire is to collect information concerning what 
parents think about the Success For All (SFA) reading program and their children's 
reading in general. Your participation is much appreciated. 

Indicate your answer for each item by checking the appropriate space. 

Age 



20-24 
25-29 
30-34 
35-39 
40-44 
45-49 
50 or above 



Gender 

’ Male 

Female 



Race 

African American 

American Indian 

Asian 

Hispanic 

White 

Other 



Education 

High School or GED 

Some college 

College degree 

Higher degree 

Other 



Indicate your agreement with each of the following statements by 
checking the appropriate response. 



1 . The school administration informed the parents about the 
Success for All program when it was initiated during 

the 1997-1998 school year. 

2. I understand the main aspects of the SFA program. 

3. I know what level my child has reached in reading. 

4. The school sends me information about my child’s 
reading progress on a regular basis. 

5. My child’s attitude about reading has improved since 
the SFA program was initiated. 

6. I like the idea of the uninterrupted 90-minute reading 
period at the beginning of the school day. 

7. I like the cross-age grouping for the reading period. 

8. My child receives one-on-one tutoring by a certified 
teacher. 
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9. I read regularly to my child at home. 

10. My child reads at home every day. 

What are some aspects of the SFA program that you especially like? 



ERIC 



What are some aspects of the SFA program that you dislike or would like to change? 
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strongly disagree 



STUDENT ATTITUDES TOWARD READING 

We would like to know your thinking about reading and books. Your participation is 
much appreciated. 

Put a check in front of the correct answer. 



Girl Kindergarten 

Boy First 

Second 



1 . 



2 . 



3 . 



4 . 



5 . 



6 . 



7 . 



Color the face that shows how you feel about each statement. 

I like having reading to start the school day. 




I read with my partner every day at school. 




I like to write stories 




I like the little books I take home to read to my family. 




My school has lots of other good books that I can take home to read. 




Sometimes a tutor helps just me learn to read. 




8. I like to tell the whole class about the books I read. 

© © 




X 



Thank you for your help! 



STUDENT ATTITUDES TOWARD READING 



We would like to know your thinking about reading and books. Your participation is 
much appreciated. 



Put a check in front of the correct answer. 



Grade: 


Gender: 


Race: 


Third 


Girl 


African American 


Fourth 


Boy 


American Indian 


Fifth 




Asian 


- Sixth 




Hispanic 






White 






Other 



Check the answer that shows how you feel about the statement. 

yes no don’t know 

1 . I like having reading to start the school day. 

2. I like being with a different group of students for reading. 

3. I read at home everyday. 

4. My school has tots of good books that I can take home to read. 

5. Reading is my favorite subject. 

6. I do a lot of creative writing at school with my cooperative group. 

7. I like to tell the whole class about the books I have read 

8. Success for All has improved my reading ability. 



Thank you for your help! 
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